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1. INTRODUCTION 



Background 

Distance Learning Project North began in September 1988 and was one of two 
developmental projects in Alberta with a primary emphasis on increasing equity in educational 
opportunity. The other project was the Distance Learning in Small Schools Pre ,ect which was 
concentrated geographically in the southeastern quadrant of the province. In a mtrast, Project 
North was located in the northwestern quadrant in a quadrilateral, the points of which are 
Meander River to the north, Slave Lake to the east, Silver Valley to the west, and Fox Creek to 
the south. Documentation provided by the Project Office 1 describes the focus of Project North 
as being to *est alternative disrance education delivery systems which can be used to provide 
equitable, cost-effective educ nion to the project schools. The first thrust of the project was to 
explor* a multi-grade model vhere a "classroom" was composed of students from a number of 
grades in a common subject area. The same document indicates that the plan also proposed that 
a multi-subject approach be undertaken in which students from a number of subject areas and 
grade levels be assembled in a distance learning classroom. Students in these classes would 
study modular, independent learning materials while receiving instructional suppon, question 
and answer support, evaluation and assessment from erf-site. A third component was to have 
been a live, interactive French program delivered v. a satellite to selected schools. 

The project was coordinated by the Alberta Correspondence School and a steering 
committee composed of senior administrators of the districts involved, together with 
representatives from other parts of Alberta Education; Digital Equipment Corporation (DEC;, 
Computer Based Training Systems (CBTS); Technology, Research and Telecommunications 
(TRT); Albert?. Government Telephones (AGT); North Peace Adult Co.^ortium; and Fairview 
College. A contract was let to Fairview College to provide a coordinator who was responsible 
for hardware md software installation, inservice, and general trouble-shooting. The 
implementation took place in stages. The first stane implemented in Fall, 1988, was the uuln- 
grade class and the subject area selected was mathematics. The Alberta Correspondence 
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Table 1.1 

Schools participant ID Yft?T Ottt flf Prtif N^rth 



County of Grande Prairie #1 


Sexsnrith Secondary 
Beaverlodge 


cast omoKy h.u. #54 


Ridgevalley 
Fox Creek 


Fairview RCSSD #35 


St Thomas \4nrp ^Pairvipw/ > 

ut« X AlV/iXjlCiO ATAVyl * ; ^1 till V1CW 


Fairview SD #50 


Hines Creek 
Worsley 


ramer Consolidated S.D. #69 


Ecole Routhier 


Fort Vermilion S.D. #52 


High Level 
La Crete 
Rainbow Lake 
Fort Vermilion Public 
Rocky Lane 

Upper Hay River (Meander River) 


Grande Prairie RCSSD #28 


St. Joseph's (Grande Prairie) 


Grande Prairie S.D. #2357 


Grande Prairie Composite 


High Prairie S.D. #48 


Kinuso 

E. W. Pratt (High Prairie) 
Roland Michener (Siave Lake) 


North Peace RCSSD #43 


Glen Mary (Peace River) 
Holy Family (Grimshaw) 


Peace River #10 


Paul Row z (Manning) 
Grimshaw Jr-Sr High School 


Spirit River S.D. #47 


Eaglesham 
Savanna 
Spirit River 
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School revised its mathematics courses to provide new independent study materials for 
students in all courses in mathematics. Digital Equipment Corporation's MicroVAX 2000 
series computers were acquired for participating sites, and a computer managed learning 
system developed and distributed by Computer Based Training Systems was installed on the 
MicroVAX 2000s to handle the functions of student tracking and testing. 

In most schools which implemented CML mathematics, a stand-alone MicroVAX 
system was installed In one case (Sexsmith Secondary), however, a MicroVAX system was 
installed that could be used remote^ , and Grand Prairie Composite and St. Joseph's in Grande 
Prairie installed terminals, printers, and modems only, and used these to access the system in 
Sexsmith. In the fall of 1989, Grande Prairie Composite discontinued taking its CML feed 
from Sexsmith and installed its ow** system. St. Joseph's is continuing to take its feed from 
Sexsmith, and Beaverlodge planning to access Sexsmith's system beginning after 
Christmas, 1989. Similarly, in 1988-89, Holy Family, Grimshaw took its feed from Glenmary 
but since then has purchased its own VAX system. (Since Summer, .989, Digital has 
discontinued making the MicroVAX 2000 series and now offers the 3000 series.) 

In the second stage of the project, implemented in January 1989, additional equipment 
supportive of distance delivery was acquired. FAX machines were acquired for all schools. 
Most acquired teleconferencing convenors, and some acquired audiographic systems. 
Teleconferencing convenors permit voice-only communication among students at partinpating 
sites which is commonly referred to as audioconferencing. Audiographic systems include the 
transmission of computer graphics as well as voice. The project provided inservice training in 
the operation of the new technology, but no formal plans seem to have been prepared for its 
implementation for instructional purposes. Several schools did try out audioconferencing in 
both instructional and non-instructional contexts. A number of obstacles arose >vhich 
prevented completion of the satellite French component of the project, and this was dropped. 
A plan to produce a series of mathematics vignettes which could be broadcast to the schools 
was also delayed. 

Participating Schools 

The number of schools involved in Project North grew during the year. Originally it 
was thought that there would be 13 participating schools. By June, 1989 the number had 
increased to 26. Table 1.1 lists the participating jurisdictions and the schools involved. 
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Table 1.2 

Year one schools and the forms of technology im plemented for instruction . 



School CML Audioconferencing 

88-89 89-90 88-89 89-90 



Beaverlodge 




l 


V 




La Crete 




5 






E. W< Pratt (High Prairie) 


2 








Eaglesham 




V 


V 




Ecole Routhier 




V 


5 


5 


Fort Vermilion Public 




V 


V 


V 


Fox Creek 


2 








Glenmary (Peace River) 4 


V 


V 


V 




Grande Prairie Comp 




V 


3 




Grimshaw Jr-Sr High Sch 4 




V 


V 




High Level 


V 


V 


V 


V 


Hines Creek 




V 


V 




Holy Family (Gnmshaw) 4 


V 


i 

V 






Kinuso 


V 


V 


V 




Paul Rowe (Manning) 4 


•I 


V 






Rainbow Lake 


V 


V 


V 


V 


KiGgevaiiey 


V 








Rocky Lane 


V 


V 


V 


V 


Roland Michener (Slave L.) 






V 




Savanna 


V 




V 




Sexsmith Secondary 


V 


V 






St. Joseph's (Grande Prairie) 


V 


V 






St Thomas More (Fairview) 


V 


V 


3 


3 


Upper Hay River 


V 


V 


V 


V 


Worsley 


V 


V 







1 . Plans under way to take CML feed from Sexsmith in winter term. 

2. Using CML materials and printouts of CML tests supplied from another school. 

3 . Supplier but not consumer of teleconferencing courses. 

4. Audiographic systems installed for future use. 

5 . Equipment present but not used for instructional purposes. 



If one examines this list one finds that it includes schx)!s of varying enrollments. Generally 
the schools have fewer than 100 high school students, though there are notable exceptions such 
as St. Joseph's in Grande Prairie and Sexsmith Secondary, and, of course, Grande Prairie 
Composite with approximately 1400 students. In the case of Grande Prairie Composite, the 
interest in CML mathematics lay not as an instructional approach for the entire school but rather 
for a very small subset of its students. One also should note i^o schools which were pan of 
the Project but in which no dis'ance education took place during the 1988-89 school year. La 
Crete and Spirit River. Table 1.2 lists the schools visited by the evaluators and indicates which 
technologies were used in these schools in each school year. Each school was contacted by 
telephone during November, 1989, to ascertain what changes had been implemented after the 
first year of the project In some cases technology which was tried during the 198°-89 school 
year was not continued into 1989-90, and, conversely, some schools implemented new 
technology during 1989-90. Where teleconferencing '<% indicated in Table 1.2, the reader may 
assume that FAX machines were employed to transmit student assignments to an off-site 
teacher, either in another school or at the Albert:: Correspondence School. (The use of the 
FAX machine in Project North posed no problems for the schools and, therefore, little attention 
will be paid the use of 'hat technology in this report. This contrasts with the evaluation of the 
DLSS Project 2 where the high volume of pages being transmitted by FAX did pose some 
problems.) 

Role of the Evaluators 

The role of the evaluators in Phase 1 of Project North needs some explanation. Because 
of the circumstances which surround the beginning of any iarge scale project, there was 
concern that the presence of external evaluators during the time when products and processes 
were just being put in place would add an unnecessary complication. On the other hand, there 
was the perception that independent evaluators might contribute information that would be 
useful to the project. Also, there was a desire to capture baseline data which might be used tn a 
subsequent evaluation of the project. Consequently the role of the evaluators during Phase 1 
was to be largely one of familiarizing themselves with the project, gathering baseline data, and 
testing evaluation methodology. They were also expected to feed back to the project leadership 
and the Planning & Policy Secretariat information whi^h might be useful in guiding the project. 
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The statement of work for the evaluation lifted four tasks ■ 

1 . To conduct an introductory tour of the sites participating in Distance Learning 
Project North. 

2. To assist in the collection of baseline data for 1988-89 and to develop interview 
protocols. 

3. ao visit each of the project sites, and meet with each of the stakeholders: project 
management, superintendents, principals, those responsible for coordinating the 
project, participating teachers, students, representatives from thr. cooperating 
agencies, and those responsible for developing the materials that were used. 

4 To produce a Phase 1 evaluation report which would include proposed revisions to 
the evaluation questions, and a straight-line projection of how the performance in 
Phase 1 might be perceived within the context of a Phase 2 evaluation. 

Because of the need to balance the provision of useful evaluative .iformation with sensitivity to 
the circumstances surrounding the start-up phase of an ambitious project, the report which 
follows will integrate the information gathered across the entire project. Perceived strengths 
and weaknesses of ir dividual sites will not be identified by location. The reader should keep in 
mind, however, that the evaluators found substantia commonality among the experiences of 
the participating schools, and the findings reported here ought nc f to be attributed to a small 
number of exceptions. 



For additional information on the DLSS Project se„, Clark, W.B. and Schieman, E., (1989) Evaluation of 
Phase 1 of the Distance Learning in Small Schools Action Research Project, Alberta 
Education; and also, Clark, W.B. and Schieman, E. (1990) Evaluation of Phase 2 of the Distance 
Learning in Small Schools Action Research Project, Alberta Education. 
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2, EVALUATION METHODOLOGY 



General Approach 

In order to obtain an accurate understanding of the implementation of the project, a 
research design which stressed the project's exploratory and developmental i._»urc »vas 
undertaken. A case smdy approach was i»sed to ensure the recognition of contextual factors 
which affected the project. This evolutionary design allowed for the inclusion of items which 
the participants considered important but which might not have been considered in the origin^ 
design of the evaluation. 

Data Gathering Techniques 

The specific methodologies employed were interviews with personnel involved in 
various aspects of the project, observation of project i^p'-mentation in the schools, and 
document analysis. The sit .s were divided between the two investigators, and over the c. *,rse 
of the year two site visits were made to each participating school. During the r *rst visit the 
principal and participating teachers were interviewed, and some in :lass observations took 
place. For the second site vLit, each invc >tigator was accompanied by two research assistants 
who interviewed students while the investigator conducted follow- jp interviews with 
teacher(s) and the principal. Confirmatory site observations *trt also conducted. All but two 
superintendents of the participating jurisdictions involved in die project were interviewed on at 
least one occasion, as were some central office staff. The investigators also attended one 
steering committee meeting as observers and spent time in intensive interviews with the project 
leaders, other Alberta Correspondence School staff, and representatives of Digital Equipment 
Corporation, Computer Based Training Systems, and ACCESS. Since the site visits to the 
schools all took place during the 1988-89 school year, telephone interviews with the principals 
were conducted during November, 1989, chie n y ^* ascertain differences between he 
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implementation cf distance education in the schools in 1989-90 compared to the previous year. 
In ail, 37 person-days were devoted to field work and data gathering. 

Interviews* The interview formats varied from a series of pre-selected questions for the 
student interviews (all instrumentation in Appendix A) to semi-structured interviews with 
school staffs and project leaders. Because of the large numbers of students who were involved 
in some schools, it was not possible to interview each student In these cases a subset of 
^jdents selected by the principal were interviewed individually, while the remaining students 
were interviewed in focus groups of seven to 10 students. Principals were requested to 
identify for individual interviews those students who represented a cross-section of the 
population. As a data gathering strategy, focus groups were of limited use. While they yielded 
useful information when suggestions for change and/or improvement were elicited, they 
proved less than satisfactory for gathering data on individual, personal experiences. Student 
data were gathered on response sheets which the interviewers filled out during the interview. 

In order to ensure consistency across the project, the investigators shared their 
experiences in using the student interview guides and response sheets after data had been 
garnered at three schools. Based on these discussions, they amended the interview scheduler 
to reflect new issues which had been identified by students. For example, in developing me 
student interview protocol the ev?luators had not anticipated that students would discover a 
variety of techniques for "short-circuiting" the way the CML system was intended to be used. 
When they met, the equators compared responses from individual and group interviews for 
richness of data and ~k 1 to continue with individual interview s wherever possible. The 
meetings of the evaluaiors and research assistants provided an opportunity to enhance inter- 
rater reliability of tne interview data through cross-checking the categorization of responses to 
individual questions. 

The semi-strrctured interviews with principals and teachers were based on common 
sets of questions drawn up by the investigators but were amended during the course of the 
conversations to follow up on topics of interest to the interviewee. The sequence of site visits 
allowed for the development of questions based on data collected a* the study progressed. 
These further questions became the focus of discussion during the second site visits. The third 
set of interviews, which were conducted b y telephone, were based on a pre-arranged i>et or 
questions that had been sent previously by FAX. 
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Observations, Time was taken at each site to visit the classroom used for CML mathematics 
and :o note rhe physical arrangements, the strategies for student groupings, the placement of 
the computers, and u;e charts, notices, binders and booklets connected with the mathematics 
program. 

Document analysis. Materials prepared for the various advisory groups and committees 
were reviewed :o ensure accuracy of contextual information. Booklets and computer- generated 
lests were examined for confirmation of items arising from the interviews. 

Data Analysis 

Student data were gathered on response sheets which the interviewers filled out during 
the interviews. These data were subsequently collated by school and then reviewed to obtain 
patterns of student responses. In order to ensure consistency across 'he project, the 
investigators maintained contact throughout the period while interviews were being conducted, 
shared their experiences, and amended the interview schedules to reflect the issues which 
arose. This also provided an opportunity to enhance the reliability of the interviewers. 
Interview data were examined after each round of visits ro identify categories of responses, 
general themes, and issues or problems which concerned the participants. The team members 
met On each of these occasions to share their individual interview notes and reflection and to 
seek frr consistency and clarity in the data analysis. A this time, other topics or questions were 
identified for possible examination. Following the second round of visits, a rough 
categorization of responses was attempted, and this was returned to participating principals. 
This had two functions. (1) It provided a means of assessing the accuracy and completeness of 
the categorized data, and (2) because it displayed the variety of responses rather than 
discussing the most comn on items, it was a potential learning device for those principals who 
wanted to know of similar concerns n other contex:s and he w colleagues had solved them. 

Credibility of Data. Data credibility and dependability were maintained through cross- 
checks with knowledgeable participants and through the use of a variety of information 
scurces, e.g., observation notes, student questionnaire data, distance education instructional 
materials, to produce triangulation of the information. Further, preliminary transcripts of the 
analysis were circulated to principals for their verification. 
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Summary 



An exploratory and iterative case study design was used to obtain data through observation*, 
interviews, and document analysis. Dan from the interviews were categorized and collated 
with document and observational information to identify achievements and concerns which 
arose vlaring the implementation and to describe the processes used to solve these concerns. 
Triangulation procedures and verification of categories and themes through checks with school 
-based participants and other knowledgeable personnel ensured the credibility and 
transferability of the data. 
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3. DESCRli xJGN OF FINDINGS 



The specific applications which formed the focus of ihls atudy vere the implementation 
of learner-centred high school mathematics correspondence school materials supported by a 
computer managed learning system, and the introduction of a v&rit.y rf other technologies to 
enhance distance learning opportunities generally. These technologies included au^ : 
conferencing, audiographics, and FAX machines. Originally, the plans foi implementation 
included the provision of a grade 10 French course which would pilot the use of interactive 
satellite broadcasts with audioconferencing as well as prepared print materials. A continuing 
problem for all the courses was the limited amount of time for course development. This was 
compounded in the French course by the fact that the provincial curriculuin was being changed 
and the new Program of Studies had not been complred. Hence, the French 10 pilot was not 
available for implementation. 

Lack of sufficient development time was also a factor in the completeness of the CML 
test bank. Although the test bank contained questions for most of the domains addressed in the 
program, the number of questions was very limited in the early stages and nere were many 
inconsistencies in the fornrt required for the student responses. Throughout the project, both 
the CML test bank and some of ii*e mathematics materials were revised to overcome those 
pedagogical concerns which were identified by Alberta Correspondence School biaff and 
participating teachers. 

While FAX machines and the MicroVAX computers and terminals were in place in rhe 
participating schools in September, 1988, the audioconferencing equipment did not ^xrive until 
February, 198? Hence, while almost all schools participated in the CML mathematics 
implementation and used FAX mach.nes to speed the turnaround time for students taking 
correspondence school courses, only a few schools piloted the Social Studies 13 course, or 
used audioconferencLj. The audiographics capability was not used mainly due to technical 
problems in set up at the sites involved. (There are plans to use the audiographics system :o 
support the teaching of Mathematics 31 between St. Thomas More High School in Fairview 
and Glenmary High School in L'eace River in tne second term of 1989-90.) 
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CML Matneinatios 

The CML maihemai :s implementation raised a number of issues for teachers, students, 
and principals at the school level. These have been categorized under the following headings. 
Because of the iterative nature of the project, many of the concerns raised by individuals in 
June, 1989 have been already addressed in this school year (1990-91). 

The Booklet. Teachers who had reviewed the booklets commented favorably on their 
co^vhensiveness in covering all topics. They recognized that all students would be required 
to cove 11 topics thereby putting them on a par with students throughout the province. Some 
coniTjemed that many assumptions about students' capabilities had not been spelled out in the 
print materials as clearly as they wished, and that such assumptions sometimes did not reflect 
the reaMty of their students' mathematical competencies. 

An examination of the booklets showed some general weaknesses, however, in their 
instructional design. Teaches and students identified these weaknesses when they requested 
more examples, greater synthesis cf content across modules, and more attention to student 
understanding of mathematical processes. Teachers also expressed concerns about student 
motivation which were linked to the vays students used the booklets. Many booklets followed 
a standa r :i format in which the general content was presented, followed by a series of examples 
to be worked on by the student. This format did not match the classroom pedagogy used in 
teaching mathematics to which the students were accustomed. In the familiar format, teachers 
worked through examples from the particular to the general, thus providing a framework for 
tht development and confirmation of principles and procedures. Only then were students given 
application items. This departure from the familiar had two impacts. (1) the student 'ended to 
s u .ip the introduction jt content and begin with the examples, and (2) teachers tried to provide 
oral overviews to units and booklets to help students focus on the important aspects. Taken to 
its extreme, some teachers used the materials as a resource only and taught their classes 
directly, reserving the examples and tests for individual seat work. 

As the booklets have been revised by ACS, a number of changes have been made, the 
pedagogy has been re-examined and the general design and visual format have been improved. 
However, there are still fewer examples to choose from than some teachers desire. The issue ot 
the quality of the pedagogical design of the materials remains a crucial factor, especially where 
teachers are not mathematics specialists and may depend on the booklets to carry the content. 

Across the project, teachers used a variety of approaches to managing CML 
mathematics. In some schools, students were required to complete the booklet before being 
allowed to access the test, while in others they did not u^e the booklet except as a resource alter 
they had completed an initial test. In units where the relationship between test bank items and 
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me booklet content was strained, some teachers gave up requiring students to work in the 
booklets since the mismatch between the practice and test items led more to confusion than 
learning. In contrast, other teachers were adamant that working through the booklets gave tht 
student a much more thorough grounding in the principles of mathematics. The teacher's 
choice of approach may also have been influenced by some of the following factors. 

Computerized testing. The paucity of items in the test bank gave rise to a number of 
problems which will be easily alleviated as the bank is expanded. In some areas, the items in 
the bank were too difficult for the grade level or did not correspond to the level of difficulty of 
the examples in the booklet. There were few items at the Mathematics 30 level which used 
wording similar to that of the Diploma examination. Some domains were tested inadequatel> 
and problem-solving items were noticeably few. 

These weaknesses, in turn, gave rise to compensatory activities by teacners, most 
frequently through the provision of additional questions. Students, recognizing the likelihood 
that consecutive tests would contain similar and sometimes the same questions, and that test 
scores were the major indicator of achievement, cho^e to u5e the tests as organizers for their 
learning. Where allowed, they drew a test, worked on the incorrect respon: es (sometimes 
using the booklets as a resource) and then drew a second test. Because computer testing was 
seen as analogous to the elimination of teacher marking, teachers were unprepared for the 
number and variety of questions which they had to mark themselves. These varied from 
booklet practice questions where the student could not find the error to test bank items where 
process was the focus. Perhaps it was the variety of questions which teachers had to handle 
within a class period w hich contributed to the teacher's frustrations with marking, since it 
meant that they had to be ab u to work in any area of the curriculum and at any grade level at a 
moment's notice. Where teachers were not mathematics specialists, this was an especially 
difficult task. Because students needed the feedback in order to know whether they could 
proceed to the next unit, some students refused to move ahead until their work was marked and 
incidentally gave themselves some spare time. In such circumstances, however, teachers 
tended to over-ride the computer's control so that students could continue. Some teachers 
commented that the provision of a pre-test would allow students to set up a challenge for 
themselves and enhance their motivation to work through the booklets. 

Another suggestion was to increase the use of the messaging function to provide a 
motivational bonus for students. Students also proposed the addition of Mathematics 30 
examples, the inclusion of Diploma practice tests, the use of a review test if a student failed a 
supervised test, and the inclusion of processes as well as answers in feedback on tests. The 
programming capability to address most of these issues was already present but perhaps 
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teachers were not aware of the suggestions or decided not to implement them in this firs; trial 
year. 

The Mathematics 30 diploma exims are a somewhat sensitive subject among schools. 
Most schools reported that students who wrote Mathematics 30 Diploma examinations in June, 
1989 did not do as well as expected. Most schools also indicated that there may be alternate 
explanations for the lower marks than the fact that the students took CML mathematics. On the 
other hand, the fact that this experience was common across so many of the schools suggest 
that the preparation for the Diploma examinations is a matter warranting close attention. 

Computer Equipment. After the initial start-up in mos* .chools, there appeared to be few 
problems with the hardware or software. Where there were difficulties, the Distance Learning 
Implementation Coordinator (Rik Hall) provided a fast, effective service. Some schools had 
three terminals hooked to their MicroVAX computer, and when all were in operation, the speed 
of operations slowed down substantially. This, in turn, created problems of access for 
students in the classroom. In general, having only two terminals on the MicroVAX was 
necessary if 15-18 students were to have ready access to the tebt bank. Digital Equipment 
Corporation indicates that new software which was loaded on the 2000 series computers now 
runs more quickly than what was available in 1989. Also, the 3000 series which are now 
available, are supposed to be much faster machines. 

Computer management Because of the limited time available for the construction of the 
test bank items, previously developed test banks were used in addition to newly constructed 
items. Unfortunately, there was a lack of consistency in the format in w\ich students were 
required to enter their answers. For example, sometimes the answer was required to the fourth 
decimal; other items required that the answer be rounded off. Where students used an 
alternative way of stating the answer, (not reducing fractions was one of the most common), 
this a^wer was not accepted as correct. Hence, in the fall term, there was much frustration 
and teacher time expended on identifying inaccurate responses, correcting the "errors" on 
individual student examinations and reclaiming student grades. The amount of time teachers 
put into these activities, and their concern that their . tudents not be penalized, cannor be 
overestimated. Teachers spent time, both in ciass *,id as much as one hour after school every 
day, changing grades and learning to operate the programs and to access student history data. 
A few teachers worked their way through all of the tests at each grade level and provided a 
binder which students could consult to see the format required. 

Many of these problems have now been eliminated, and most teachers feel reasonably 
"competent" with the system; however, the diagnostic functions of the computer program are 
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generally under-utilized and teachers are only beginning to understand how they can be 
integrated into their program. Few teachers use the program to identify areas of student 
weakness so that they can provide small group remediation. This is one area where teachers 
can use the computer to do the initial selection rather than depending on their own memory of 
student errors. A manual which would outline these possibilities and show the programming 
nec^sary to acce s these functions is new in preparation. 

Scheduling. With respect to scheduling, schools varied along three dimensions. (1) They 
varied in the extent to which they formally scheduled mathematics periods, rangi.. b irom 
schools with no designated periods, to those with some assigned periods, to th se with all 
assigned classes. (2) They also varied .n the composition of these classes where they ranged 
from classes of mathematics students from all grades to those with specific combinations of 
grades, to those designated by grade level, e.g., 10's. (3) Class size was the third variable 
with groups ranging from a high of 34 to those with 5 students. 

These three factors combined to form variations not all of which were equally 
successful. Students and school staff generally preferred at least some scheduled times. They 
also found it best to limit classes, thereby reducing the range of possible tutor topics for the 
teacher. Some did this by confi ring the class to one grade level while others combined no 
more than three different learning combinations (e.g., 31 , 23, 10). The larger the class size at 
one grade level, the more likely it was that the teacher had to group the students rather than 
work v/ith them individually to provide assistance. The number of students doing each 
variation of mathematics at that grade level was also a factor. There was some indication in 
Phase 1, incidentally, that a maximum of approximately 20 students (some would say 16) 
seems to be an appropriate number until the procedures for assisting students become more 
streamlined md the speed of computer test generation quickened. After their expe. .ences in 
1 988-89, most schools are much more aware ot* the potential problems raised by each 
combination and have taken steps to ensure a reasonable class in size and grade diversity. 

While the fluidity in enrolment of grades 1 1 and 12 continues to be an uncontrollable 
factor for small rural high schools, principals now are able to offer students the opportunit> to 
obtain credits regardless of timetable or employment conflicts. Many principals spoke of 
individual arrangement which had been highly beneficial to the students and which the school 
could not have accommodated in any other way 

Learning management. The specific learning management structures which teachers 
implemented reflected most strongly the philosophical orientation of the introductory inservice. 
These teachers had heard that they would be facilitators of learning who provided individual 
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tutoring to students learning at their own rates. (Some believed that the CML system was 
actually a computer assisted learning or CAL system.) They began the year b> spending tinn, 
discussing student responsibilities including the sequence of the course content, the numov. * 
modules, the use of the computer as a testing tool and the importance of working consistently 
to finish the course in the time provided. Many developed a time line based on co „uOn dates 
for the units at that grade level and made posters identifying these dares so that students could 
easily calculate the difference between where they were and where u.ey were supposed to be. 
Because the provision for individual student progression which was emphasized as a core 
value of the program was one most teacheis espoused, they did not intervene and enforce 
deadlines by which students had to have completed each unit. The notion of "going at one's 
own rate" was taken to apply to classroom time. In fact, many students did no homework and 
did not use their spares to meet the unit dates necessary for course completion. While some 
teachers foe iwi on keeping students on task and introduced the topics of the bookiets in mini- 
lessons, others were besieged by forests of hands and spent their entire periods answering 
questions and marking or remarking tests and examinations. 

Of importance was whether :eachers demanded that booklets be completed prior to 
drawing a test. Some teachers just advised students to do the booklet while others gave a grade 
f or the completed booklet; some allowed students to pull a test but wanted to see the completed 
work : n the booklet before assisting students with their mistakes. Other teachers started by 
requesting completion of booklets and then found themselves and their students frustrated by 
the lack of congrvity between the booklets and the tests for some units and consequently gave 
up the procedure. Where teachers were working with 20 or more students, supervision of 
booklet completion was onerous and time consuming. In many instances, students were 
unwilling to proceed unless the teacher had marked the booklet or the test exercises. This le< 
to unstructured time for the student and a potential source of disruptive behavior in the 
classroom. 

Unwilling to disrupt the option of "own rate" progression, some teachers used grading 
schemes to structure s*uder:s' work. These schemes included allocating marks for completing 
*he booklets or counting the mark received on the first of the practice exercises toward the 
course grade but still requiring 80% in three tries for competency. Other teachers assigned a 
percentage of the final mark to each test. All these procedures were designed to discourage 
students from drawing tests before attempting the oookiets. Where students were required to 
complete the booklets, there were fewer complaints about iht lack of test/booklet correlation. 
In some instances, teachers pulled review tests prior to the teacher supervised test to help 
students review several booklets at once. 
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Teacher assistance. Where teachers had classes of 26 or more students, o* where all 
students who had spares had the option of attending the mathematics room, teachers had 
difficulty meeting all the students' requests for assistance. One school had provided and others 
planned to provide, a student with good peer tutoring skills who bad completed Mathematics 
30 and could earn wc;k experience credits by providing student assistance. In another school 
where the Micro VAX was heavily utilized, an adult w&, employed to enter student test 
responses into the CML system and thereby reduce the lag time while students waited for 
access to the computer. While it solved the "wait" problem, it also denied the student access to 
the computer and the immediacy of response which was a positive motivational feature for 
some students. In one school, the adult aide, at the request of the teacher, used the computer tu 
obtain diagnostic information from the test results. From this information, the teacher then 
identified ihe small group of learners who needed further work. Tasks for assistants varied 
from non-instructional activities such as entering tests and pulling diagnostic information and 
student histories *o situations where the focus was on individual student assistance and where 
the aide identified for the teacher those students with a common concern. 

Classroom environment Not all teachers seemed equally aware of the potential impact of 
the classroom's physical environment. The suggestions given tr teachers at the preliminary 
workshop provided some options, but most often the location of computer plugs and telephone 
jacks were significant factors in room organization. Some teachers clustered desks or tables in 
an oval or horseshoe format with the teacher's desk at the open end while others had tables or 
desk clusters placed throughout the room. 3ome formats made teacher supervision more 
difficult. Having a space set aside for the teacher supervised tests was also important in 
helping the teacher control the situation in the classroom. This \s another area where evident 
changes have been made this year, following the experiences in 1988 89. Where the social 
organization of the class is not controlled by direct focus on the teacher's voice and actions, 
then the physical structure of the classroom is important in conveying the message of a 
common foct ,c on task completion. 

Cooperative learning. In many schools, students ueveloped informal learning groups 
whose members provided support and assistance to each other Many teachers commented 
positively on the increase in peer tutoring and in task-focused conversation. They noted thai 
where students formed supportive work groups, they were more likely to keep to the schedule 
and because they all worked on the same booklets, it was easier for the teacher to provide a 
mini-lesson to introduce the concept and ensure that it was clearly unders.ood. A small number 
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of teachers indicated that the conversation in such groups tended to be more social than 
academic. 

Tutoring-Teaching. From he initial inservice, teachers clearly expected that their time 
would be taken up in tutoring individual studer ts rather than in didactic teaching. One teacher 
:ommented after this introduction, "This is management, where is the teaching?" Despite a 
desire to go over the concepts and explain them in order to ensure that students fully 
understood the topics, teachers in general avoided full class lectures. 

The transfer of students from a system where the teacher provided the impetus through 
a class lecture, and sustained the focus through seat work, to one whe . students had to obtain 
all the information from written booklets and provide their own motivation with limited teacher 
assistance was a difficult transition for many students. Some teachers felt that introductory 
sessions were essential to ensure students had the necessary background and motivation but 
found these difficult to continue as students spread out through the units. 

There were difficulties at the beginning of the school year in both 1988-89 and in 
1989-90. In September, 1988, this was due to the late arrival of tb* print materials which 
came in mid October. There was a similar delay in Fall, 19S9 because, following revisions, 
there was insufficient time to produce enough booklets and the numbers of schools requesting 
materials escalated. Fortunately, the delay was not as long as in the previous year. 

Most teachers confined themselves to working with individual students. In organizing 
for this school year,however, mo^t planned to do mors direct teaching. Some thought that they 
would introduce new concepts to the entire class as the bulk of the students began a new unit, 
with slower students taking notes whk i the teacher would go over laier, and with faster 
students using it as a review. Some planned to keep students in cohorts so that they ^ouiu 
monitor them more closely, ^me *?lked about dropping th* three-pan competency provision 
and replacing it with a once-only test. While these options would seem to address a number of 
issues — keeping students moving forward at a reasonable pace and ensunng that all students 
had a full understating of the concepts — they would limit the full benefit of the program, 
individual student progicss. fhe drive to include direct teaching, either to small groups or to a 
total class, did not result so much from teachers' desire to teach as from a concern that the print 
materials did nc t introduce the topics appropriately. There was a concern that weaker students 
in particular, * ere learning to focus on mathematical procedures rather than principles, and that 
students did not see connections or were unable to address problems if they were stated in a 
format other than the one they had worked on in the booklet. Because the Mathematics 30 test 
bank in 1989 90 did not incmde questions in the same format as appeared in the Diploma 
examinations, many teachers did intensive coaching with students to ensure that they wt: 



comfortable with the question structure and felt prepared for the examination. If teachers were 
convinced that students' needs were well served, they might then find ways to encourage 
students to maintain a steady pace other than by direct intervention in their learning. 

Student Motivation. This was a topic of major interest for school staffs and students alike. 
All felt that students cf whatever aoility did well using this format if they were self-motivated. 
Their major concern was for the unmotivated students — especially those who didn't like eithei 
mathematics or reading. Without a support structure to guide their learning, many students 
were unable to cope with their newfound freedom. Some found the first two units relatively 
easy, and they relaxed, believing that catching up wculd not be difficult. Others gave up when 
they found that the format of the units was similar, and boredom set in. For others, the amount 
of reading required was their undoing. These students were used to an "oral/ visual" pedagogy 
and found the transfer to reading tu be difficult since it involved another level of abstraction. 

These poorly motivated students provided the biggest challenge and concern for 
teachers. They used up teacher time because they needed the longest explanations and yet were 
readiiy distracted if the teacher was working with another student Some found ways to "beat 
the system" by pulling tests, comparing answers and seeking nelp from others. Only as a last 
resort, did they consult the booklets. Asked to explain what they did if they did not pass the 
competent test the first time, t'.ie majority of these students immediately drew a second test.. 
In compai. *n, however, some motivated students used the system to accelerate their progress 
and complete two or more mathematics courses in one year. These students generally enjoyed 
being able to pace themselves. 

Reporting-Monitoring. Most teachers began by using the grade guide in the booklets and 
then amended it when they encountered problems. Teachers were somewhat frustrated because 
they felt responsible for the academic achievements of their students but, because students 
controlled the pace of their learning, saw themselves as unable to require activities which 
would ensure at least a minimum level of competency. "Working at one s own rate" meant 
that, in general, teac\ers monitored which units students were on, but only when little or 
nothing was accomplished over the course of a reporting period, did they approach the 
principal. 

Teachers found the management function of the computer program very helpful *n 
keeping track of students' progress. While some teachers pulled the information weekly, most 
collected histories about once per month. Some shared these with their students and 
encouraged them to show them to their parents, but teachers were aware that this did not 
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always happen. This was >ne feature which was used heavily at the beginning ^ .ne schoo, 
year but was gradually ignored as the amount of marking inci eased exponentially. 

Many teachers knew which units students were on but did nc: always keep track of 
how Ions they had been working on that unit. While some teachers sent home to parents 
regular histories reporting the unit the student was on and the marks for the competency icsts to 
date, others also included expected progress' information so that parents knew what was 
expected. 

The issues of course failure or continuation have been variously addressed. Where 
students were a unit or two short of course completion, some teachers planned to give a review 
test in Septeml^er and then allow the student to continue from that competency level. Others 
encouraged selected students to work on their booklets over the summer. In fact, students who 
did n3t complete are continuing their courses either as a regular member of their scheduled 
class or in their spare periods. Happily, many students who did little work on the course last 
year have already completed that course and are working to catch up with their classmates. 
Students who had only one or two units completed, were counselled either to work at a lower 
level or repeat the enure course. This has not been a difficulty. 

Audioconferencing 

Audioconferencing was not heavily used during Phase 1, but a number of schools did 
employ it with a small number of students. Courses which were supported at least in part by 
audioconferencing were Accounting 10, Business Education 20, Biology 30, German, 
Mathematics 31, and Physics 20. 

Whereas the introduction of the mathematics program occasioned teachers to reorga n ise 
the ways they structured their teaching, audioconferencirg called for the development of 
teaching strategies which were not a regular pan of their teaching repertoire. Accustomed to 
monitoring classes through non-verbal cues and using the jhalkboard as a focus for didactic 
instruction, teachers had to depend on print materials to a much greater extent. Those who did 
not have prepared correspondence materials had to develop such materials. But, not only had 
teachers to obtain or develop materials, they also had to learn how to discipline their own on- air 
time znd pace the instruction, to vary their instructional strategies to include a greater variety of 
voices and activities than is necessary in the classroom, and to n onitor student learning and 
engender discussion using verbal cues alone. 

Audioconferencing also requires appropriate hardware set up in convenient locations 
conducive to learning. Some schools had not given sufficient thought to the ramifications of 
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location and had put the telephone hook-up, the fax machine and the CML terminals beside 
each other. Another important consideration was tlie cost of long-distance telephone calls. 
Especially at the high school level where these schools are likely to be geographically far apart, 
inter-school cooperative teaching means regular long-distance costs. 

Audioconferencing .vas most effective in providing tutoring for students who were 
doing Alberta Correspondence School courses. Ln these instances, the teacher called the 
students and dealt v \th any general questions first before providing individual assistance. 
These sessions were usually about 30 minutes long for 4 or 5 students and did not entail the 
need for a supervisory teacher at the receiving site. Individual students occasionally called the 
teacher for help at other times. Tnis year, 1989-90, a number of the schools which tried audio- 
conferencing in Phase 1 are noi using it. However, schools are beginning to use the equipment 
to enhance regular classroom instruction, such as "bringing in M guests to the class by 
telephone, and for informal student cou..w*i meenngs between schools. In one school district, 
Mechanics 12, Accounting 30, Physics 10 and French 10 a^e being offered by 
audioconferencing. These experiences will prove valuable in identifying those skills wb'ch 
teachers need to address if the system is to be successful for learners and teachers. It shjuld be 
noted that some schools which empioyed audioconferencing last year are not doing sc Ms 
year. The reasons associated with dropping it appear to be related ;o the managemeii J 
pedagogical issi es noted above. In the schools which nave ^isconunued it, theie was also a 
pattern in which (1) the teachers were doing tne audioconferencing courses in addition to their 
regular class assignments with no extra remuneration and (T were tutoring the subject, but the 
student assignments were being given and marked elsewhere. 



Level of Satisfaction 

The introducnon of CML mathematics ha? u ~en generally welcomed by principals and 
teachers because u provides for options which would not be possible in any other vay. Not 
only does it expand the number of courses available, but more importantly, principals are now 
able to offer students coupes more suited to their abilities. Formerly, in a school v.rh seven ur 
eight Mathematics 30 students and two more suited iO Mathematics 33, the teacher had eitner to 
accommodate the materials to the range of ability levels and hope tht students cou*d manage to 
keep up or suggest that they work on correspondence mattnals. Most often such students did 
not take mathematics classes at all. Much of the student population :n the northern Alberta is 
transient, and through the use of correspondence school materials, principals now have a 
means of helping students complete courses for credit rather 'han having them lose ;i.eir 
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semester's work. Besides the oiganiza&onal benefits, principals and teachers were also 
pleased at the growth in maturity as students took responsibility for their own learning and 
showed enthusiasm for their work. Asked if they would recommend the CML approach to 
others, students talked about the importance of motivation and being willing to work with little 
direc. assistance. Thirty-six percent recommended it outright, and another 32 percent agreed, 
with the addition of the motivational caveat 

Audioconferencing has been used by too few teachers and students to have been fully 
explored. Where it has been used for tutoring, students, teachers and principals are pleased 
with the opportunity it affords to provide assistance to students. Its use as a technology to 
provide regular classroom instruction is in the process of being tested this year. 

Summary 

In general, the organization and structure necessary for successful implementation of 
CML mathematics has been identified and discussed in the categories identified above. Of 
continuing interest are the pedagogies which arise from using the correspondence materials 
within the regular classroom. In every case where the instruction is based on correspondence 
materials, the importance of the pedagogical design of the booklets and associated media cannot 
be over-emphasized. 
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4. RECOMMENDATIONS & COSTS 

In many ways the findings reported in Chapter 3 arc themselves the recommendations 
from this evaluation. There are, however, certain global areas upon which the evaluators wish 
to comment 



Materials Design 

First, the evaluators commend the Albena Correspondence School and all those who 
worked on developing and revising the materials for their responsiveness to feedback from u*w 
field The mau natics course materials in the schools in 1989/90 were an extensively revised 
version of those in use in Year one of the project. Based on Year one data, a small number of 
students commented (a) that they found the printed materials followed a predictable pattern 
which over a number of lessors became monotonous and (b) that efforts at humour in some of 
the booklets occurred in places which, rather than adding levity, caused them frustration. A 
number of teachers commented that they found the materials strong for teaching procedures but 
weak for helping students identify and understand the underlying principles and recognize then- 
potential application in new situations. Teachers also commented that the approach taken in 
presenting new content differed substantially from that to which students were accustomed. 
From such perceptions and a general review of the findings, the following recommendations 
are drawn: 

1 . The continued up-grading of the print materials to delineate learning objectives, 
cognitive linkages, requu"ed background knowledge and competencies, and evaluation 
options, as well as closer attention to a variety of learning strategies for differing abiliry 
levels is applauded and should be extended to all distance learning courses. 

2. Developers of instructional materials should be sensitive to the fact that research has 
shown that instructional materials which follow a predictable pattern tend to be 
demotivating, and should vary the presentation form. 
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3 . Humour should be used judiciously, be relevant to the subject matter, and be placed at 
points where students can be expected not to be experiencing frustration. 

4 . Curricular goals which emphasize principles or the learning process should be explicitly 
identified in the printed materials; and content addressing them should be labelled as 
such. 

5 . The materials should be examined to see if the introduction of new content indeed 
moves "from the known to the unknown", "from the concrete to the abstract" Since 
each content area requires particular pedagogical strategies, these, too, must be part of 
the course design. 

6 - Students should be oriented to the organization of the printed materials and taught how 
to use them effectively. 

In addition, the evaluators would make seven other suggestions pertaining to the materials: 

7 . Instructional materials should be prepared which assist students to synthesize 
information across topics rather thai depending upon tests to cause synthesis to occur. 

8 . The effort to increase the size and comprehensiveness of the test banks should 
continue. Further, given the pressures on non-specialist teachers to be able to answer 
inquiries about all aspects of the curriculum, there should be greater emphasis on Lhe 
development of an expert system which would identify the sources **? assistance 
available in the course materials and textbooks. Such a system might also extend the 
ime the teacher has available to assist students since it would alle /iate some of the more 
routine student questions. 

) . The reality o c the Departmental exa minat^ns should be recognized and provision made 
to prepare students for response formats which presently cannot be keyed into the CML 
system. 

10. Non-print resources that can be used to augment the courses and reduce dependency on 
print-based material should be acquired 
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1 1 . Since the course materials must correspond to the Provincial Curriculum, it would seem 
advisable that there be a close working relationship between ACS and pzT>vinciaJ 
curriculum development initiatives not just in the initial development of a course but 
also when changes are envisaged This would enhance the likelihood that the distance 
learning materials would accurately reflect curriculum intentions and that particular 
requirements which impact on distance learners could be addressed 

1 2. The planning of appropriate teaching strategies by distance learning teachers would be 
enhanced if they received a design overview for each course. This should delineate for 
each unit or module,the specific intentions, content knowledge and instructional 
processes which flow from the Program of Studies and how these relate to each other. 
Such an overview would provide branching options to teachers and learners. It also 
provides a student or teacher with easy access to specific topics for review or further 
study. 

13. Given the limited use of audioconferencing, with or without the au biographies system, 
further consideration needs to be given to assisting teachers to use these technologies 
for effective instruction. In particular, print materials which would guide the teacher in 

z preparation of materials, the use of instructional strategies, and appropriate 
Iministrative arrangements aie needed The necessary administrative arrangements 
i long distance costs may also be limiting factors in the effective use of this 
.pnoiogy. 

Teacher Inservice 

The transition from the traditional classroom to the use of computer managed learning 
and distance education techniques is not necessarily an easy one for all teachers, and the 
methods nece:*ary to assist students and manage classes in this new environment are not 
intuitively obvious. Consequently inservice for teachers involved in distance education should 
be extended beyond training in the use of the equipment to include (1) pedagogical techniques 
appropriate for different subject areas and , (2) appropriate configurations for distance learning 
classrooms, and (3) classroom management techniques. Resources such as the Distance 
Education Inservice Kit developed for Alberta Education have been prepared to assist the 
inservice education of teachers who will be involved in distance education. Besides supporting 
workshops for teacheis who will be involved in this mode of education, Alberta Education 
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should request that the Faculties of Education in Alberta Universities begin to address 
pedagogy in distance education in both their inseivice and pre-service programs. 



Cost Effectiveness 

Since no formal cost effeptiveness anal> sis was performed during Phase 1 , the 
following is offered as a somewhat terse analysis of costs for those who may be interested in 
adopting CML mathematics. Except where aides were engaged to assist with large CML 
classes, the i r omental costs associated with CML were associated with the capital costs of the 
MicroVAX system and the CBTS software, and the cost of the student materials. All of these 
costs have changed since Phase 1. The MicroVAX 2000 series has been replaced by the 3000 
series. The cost of the latter as a stand-alone system is quoted as $23,321.30. The CML 
system leased from CBTS is now available to the schools under a provincial license to Alberta 
Education. The cost of the Distance Education student materials for a 5-credit course is 
proposed to increase from $2C per course to $50 per course. Assuming an amortization period 
of five years for the MicroVAX, the annual cost of the system would be $4,665 plus 
maintenance costs. The present approach to maintenance appears to be that of acquiring an 
additional system for clusters of schools, and substituting parts from this system for parts 
which malfunction in the school systems. The malfunctioning components are then repaired by 
Digital on a longer turn-around, lower-cost basis. Again amortizing the cost over five years, 
this arrangement probably adds another $1,000 per year to the cost for each school. Finally, a 
cluster of schools requires a coordinator whose responsibility it is to install equipment, conduct 
inservice training, and the like. In 1989-90, this was estimated to cost $75,000 for 26 schools, 
or approximately $2,900 per school. This brings the average cost per school per year for 
equipment and coordination to $ 8,565. 

The per studeat co^ for the equipment decrease with an increase in the number of 
students, and for supplies increase with an increase in the number of students. Following are 
some sample figures expressed as cost per FTE Students, where an FTE student is equal to 35 
credits, and 5 credits per course is assumed: 
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Table 4.1 

CML pgr suidgm costs for different numbers of students. 



of Students No. of fte Students Costper FTP Stude nt 



25 3.6 $2726 

50 7.1 1558 

75 10.8 1'40 

100 14.3 949 

125 17.9 827 



The cost column in this table was derived by multiplying the number of students by the cost of 
one 5-ciedit course ($50), adding the amortized capital costs plus maintenance and coordination 
costs, then dividing by the number of FTE students. (No provision was made for additional 
support staff such as a teacher aide.) 

These cost estimates dQ include the capital cost of equipment acquisition. School 
districts may choose not to include equipment costs in calculating the incremental cost of using 
CML. In that case the per FTE student costs would be considerably lower. However, school 
districts should be mindful of the fact that data processing equipment has a relatively short life 
expectancy. They nm. b prepared to upgrade or replace the system when it no longer meets 
the demands required of it. 

Cost data were not gathered for the courses in which audioconferencing took place. 
There do not, however, appear to have been any additional staff costs associated with 
audioconferencing. At this point, more needs to be done to make the audioconferencing work 
well before it is appropriate to conduct a cost analysis. Schools did note that long distance 
:harges associated with audioconferencing between some schools was considered a. significant 
disadvantage if the school had to pay the charges. 
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APPENDIX A 
INTERVIEW PROTOCOLS 



PRINCIPAL INTERVIEW PROTOCOL — DLPN 



1 . What elements of DL are in your school? CML, TC, ACS 

2. From the school's perspective what, if any, has been the impact on . . . 

2.1. ... student program 

2.2. . . . staff 

2.3. ... facilities 

2.4. . . . time-tabling 

2.5. . . . resources not funded by Alberta Education (e.g., libraries) 

2.6. ... relationships to other schools and districts 

2.7. ... school finances 

2.8. . . .other? 



3. 1 V hat assistance from the District did you receive in implementing the program? 

3.2 What additional assistance do you think you ought to have received? 



4. Satisfaction: 

4. 1 . What level of satisfaction do you perceive your staff to have with the Distance 
Learning project? 

4.1.1. What about those not participating directly? 

4.2. How have the parents of your students reacted to the project? 
4.2. 1 How have parents been informed about the project? 

4.3. What community reaction has there been apart from parents of the students 
involved? 

4.3.1. What indicators have you had? 

4.4. About yourself, how satisfied have you been with the project to date? 

4.4. 1 . Has the staffing within your school worked the way you hoped it 
would? 

4.4.2. Can you comment on the relationship between your school and the Off- 
site teachers? 

4.4.3. How has the project been coordinated in your school? 



5. Expectations: 

5.1. What benefits did your school expect from participating in the Distance 
Learning project? 

5.2. What benefits do you feel your school ohr.ined? 

5.3. What was expected of your school to participate in the program? 
5.3.1. Was your school able to meet those expectations? 

5.4. What was expected of you personally 0 

5.4.1. Were you able to meet tho^e expectations? 
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Tell me how you went about allocating resources (e.g., staff) to the Distance 
Learning project and why. 

Let's talk about scheduling for audioconferencing or live ACCESS broadcasts... 

Has communication with the project leadership been adequate for you to 
operate the program satisfactorily? 

8.1. Do you have suggestions for changing the nature of communication? 
Strengths & Weaknesses: 

9.1. What would you identify to be the strengths of the project in its present form? 

9.2. What would you identify to be the weaknesses? 

What would you suggest as the best way to organize and manage the distance learning 
projec' across the participating schools? 

Generally, do you have suggestions for changes you would like to see in the 
program? 
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TEACHER INTERVIEW PROTOCOL 



1 . Expectations 

1 . 1 What courses have you been involved with in DL? 

1 .2. Tell me generally what your role has been in the distance learning courses? 

1.3. Now let's review the details of what you have actually been doing. 



[1] 


Tending fax 


[2] 


Record keeping 


[3] 


Tending phone 


[4] 


Tutonng/teaching DLSS subjects 


[3] 


Administration 


[6] 


Maintenance of hardware 


7] 


EMail 


8] 


Computer Con f erencing 


[9] 


Computer Managed Learning 


[ 1 


Other 



1 .4 Did what you originally expected to do differ from wh:u you have actually 
done? [ | Yes [ | No 

IF YES, in what ways? 

1 .5 How were you prepared to work with distance '.earning? 

1.6. Have problems arisen in implementing the project that you didn't expect'/ 
If yes, specify. 



2 . Time 

2.1. What have been the demands on your time as a result of DL? 
2.2 If additional ri* ^e, when is it spent? 



3 . Changes in Role 

3.1. Have there been changes in your role as new technologies were introduced? 
3.1.1. EMail * yes [ ] no [ ] 

3.1.2 Computer Conferencing yes i ] no [ j 

3.1.3 ACCESS Broadcasts yes f ] no [ ] 

3.1.4 Computer Managed Learning yes [ j no [ I 

3.2. If there have been changes, what is the nature of the change? 



4 . Colleagues 

4. 1 . How functional has been the relationship between you and the off-site 
teachers? 
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4.1.1. Tell me about some of the things in your relationship with the off-site 
teachers that you were pleased with. 

4.1.2, Tell me some of the things that you perceived as dysfunctional — didn't 
work well. 



Tutoring 

5.1. How much tutoring took place locally? 

5.1.1. Who is doing the tutoring? [| coordinator | | aide 
[ ] other teachers | ) other students? 

5.1.2. IF BY OTHER TEACHERS, was it [| casual I J frequent? 

5.1.3. IF BY OTHER TEACHERS, was it [] formal [] informal? 

5. 1 .4 IF BY OTHER STUDENTS, was it { ] casual [ ] frequent? 

5. 1 .5 IF BY OTHER STUDENTS, was it ( ] formal ( ] informal? 

5 2. How frequently were the tutor/markers called upon for tutoring assistance? 
ACS Materials 

6.1. What do you see as the strengths of the ACS Materials? 

6.1.1 Old ACS Materials 

6. 1 . 1 . 1 Please give examples 

6.1.2 New ACS Materials 

6. 1 .2. 1 Please give examples 

6.2. What do you see as the weakness of the ACS Materials? 

6.2,1. Old ACS Materials 

6.2. 1 . 1 Please give examples 

6.2.2 New ACS Materials 

6.2.2.1 Please give examples 

Learning/teaching issues 

7. 1 What has been the impact on student learning, and has it been differential? 

7.2 How has student progress been tracked and managed? 
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Supporting resources (Note courses referenced) 

8.1. How have your physical facilities helped or hindered distance education? 

8.2 Are there material resources which you found necessary but lacking? 
7.2.1 EF YES, please give examples. 

8.3 Did you use human resources other than those formally available through the 
program (e.g., persons in the community)? 

8.4 Were there human resources for distance learning which you would have liked 
but which were unavailable to you? 

7.4. 1 IF YES, please *ive examples. 



Suggestions 

9.1. What resources do you feel need to be added to improve the program? 

9.2. What suggestions would you have for improving the coordinator role? 

9.3. What other changes would you suggest? 
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STUDENT INTERVIEW PROTOCOL 



1 . First, tell me which courses you have been taking through Distance Learning this yeai. 

2. Have you ever taken correspondence school courses before? [ ] Yes [ ] No 

2.1. IF YES... 

2.1,1. When? 
2.1.^ What courses? 

2.1.3. What did you find different between those courses and the ones , iu are 
taking through Distance Learning? 



3. When did you do most of your work for your DL courses? 

[ ] Regularly scheduled periods | | Spares ( | Outside school hours 



4. Where did you do most of you* - work for your DL courses? 

[ ] At Home [ | DLRoom [ ] Library [ ] Regular classroom 



5. Is where you study and when you study for the DL courses tne same pattern as for 
your "regular" courses? 

[ | Yes I 1 No 

6. How do your grades compare bet vveen DL Courses and courses locally available in 
your school? 

CML [ | DL better | ! DL the same I | DL poorer 

ACS [ | DL better [ | DL the same | | OL poorer 

TC [ | DL better | | DL the same | | DL poorer 



7. When you were studying and needed help. . how did you get it? 

[ | Didn't get help [ | Aide [ | ACS Marker [ ) Subject Teacher 
( j Other Teachers [ ] Other Students [ | Parents [ ] Other 
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8. Which of the following resources did you use, if an"? 



library books 
films 

videotapes/cassettes 
filmstrips 

audiotapes/cassettes 
kits 

teachers in your school 
Tutor/Markers 



Perceived Usefulness 
¥m Somewhat 



I 1 
( 1 
[ ] 
[ ] 
( ] 
[ 1 
[ ] 
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Let's talk about the different components of Distance Learning? 

9.1. OLD Correspondence Course Material 

9.1.1. Was the reading level [ | too easy, | (about right, | | too Jfficult? 

9.1.2. Was the way the material was organized { j easy to use or 
[ ] lifflcult to use? 

9.1.2.1. Why? 



9.2. NEW Correspondence Course Materials 

9.2. 1 . Was the reading level [ ] too easy, ( 1 about right, | 1 too difficult? 

9.2.2 Was the way the material was organized [ j easy to use or 
[ ] difficult to use? 

9.1.2.1. Why? 



9.3. FAX 



9.3.1. Was the turnaround time getting feedback to you fast enough 9 
I ] Yes [ 1 No 

9.3 2. Was :he feedback you received useful ic vou? 
( 1 Yes [ ] No 

9.3 2. 1 . Did you normally read the feedback you received? 



9.3.3. How wns the quality of print on the faxed materials? 



ERIC 
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Computer 

9.4.1 Did you send/received messages from the Tutor/Marker using 
electronic mail? I ] Yes [ j No 

9.4. 1 . 1 How useful did you find electronic mail? 

9.4.2 Did you use the electronic bulletin board? 
| ] Yes ( ) No 

9.4.2. 1 How useful did you find the electronic bulletin board? 



Audioconferencing 

9.5. 1 . Did you participate in audioconferencing sessions? 
i ] Yes [ ] No 



9.5.1.1. IF YES. . . 

9.5.1 1 1. Which courses? 

9-5 1.1.2. Did you feel comfortable talking? 
IF NOT, why not? 



9.5 1.13 Could you easily understand what was being said' : 
I I Yes || No 



Computer Managed Learning 

9.6. 1 . Did you use a computer managed learning svstem? 

I jYes [ 1 No 

IF YES.. 

9.6.2. Which course(s) did you use it in? 

9.6.3. Tell me what, if anything, you liked about it. 

9 6.4 Tell me what, if anything, you disliked about it 

9 6.5 Did you find the computer easy io use } \ \ YES ( | NO 
9 6 6 What was the role of your teacher in the CML coursefs^ 
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9.7. Off-site teachers (ACS or another school) 

9.7. 1 . Did you get useful feedback on your assignments? 
[ ] Yes [ ] No [ ) Sometimes 

9.7.1 ! . Give me some examples that you can think of. 

9.7.2. Did you ever contact a tutor at ACS or in another school directly 
looking for assistance? [ ] Yes [ ] No 

9.7.2.1. IF YES... which of the following did you use? 

[ ] phone 

[ 1 FAX 

[ ] electronic mail 

[ ] electronic bulletin board 

9.7.2.2. IF YES...did you get the help you needed? 
[ ] Yes f 1 No 

9.7.2.3. IF YES... were you looking for help with course 
material or for advice on other matters, for example, 
what course to take next? 

^ What would you be doing with your rime if Distance Looming weren't availabl e? 

11. Did you find it difficult to get assignments done on time? f 1 Ye* f 1 No 
12.1. IF YES...why? 
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For courses where there wasn't a iocai teacher available, does it matter to you that there 
wasn't a local teacher for the DL courses you took? 

Did you find that you needed materials that weren't available locally? 
[ ] Yes [ | No 

13.1. IF YES., -nat kinds of material? 

13.2. Were you able to get them? [ J Yes [ ] No 
13.2.1. IF YES...huw and with whose help? 

What did your school do to prepare >ou to take courses through Distance Leaning? 



1 5 If yc* could make a.'inges in the program, what changes would you like to see? 

1 6. ■ jiven your experience »hus far ^nd choice in the matter, would you continue to take 
routes this way? 

i 7 would you recommend to other students that they take courses this way? 

That's the end of my list. Have you got any questions for me 0 
Thanks very much for helping. Good luck. 
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QUESTIONS ADDED TO STUDENT PROTOCOL: 



1. Did you use the computer to get your own tests? 

2. Did you have any difficulty using the computer? 

3 . Was the level of difficulty of the tests: OK? too easy? 

4. Did you complete the booklet before yeu toon the test? 

5 . What did you do if you did not get 80% on your first try 

Drew second test? 
Reviewed booklet? 
Did booklet again ? 
Reviewed test? 

Sought help — teacher? peer? 
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